Data processing and storage systems that have to address thousands of simultaneous transactions within tens or hundreds of milliseconds face substantial challenges in scaling while keeping latency at a minimum. These difficulties increase when the results of data processing operations depend on data that is continuously collected from many remote clients. Accordingly, there is a need to provide real-time data processing systems with high-performance data storage and streaming architectures that address the unique problems arising in these types of systems at large scales.